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COMMENTS  ON 


"THE  MEASUREMENT  OF  LINEAR  DEPENDENCE  AND 
FEEDBACK  BETWEEN  MULTIPLE  TIME  SERIES" 

BY  JOHN  GEWEKE 

by  EMANUEL  PARZEN 

Institute  of  Statistics 
Texas  A&M  University 

I  would  like  to  congratulate  Professor  Geweke  on  an 
interesting  paper.  I  believe  its  most  valuable  contribution 
is  to  stimulate  us  to  develop  improved  methods  for  modeling 
multiple  time  series  with  the  aim  of  determining  which  variables 
are  significantly  related. 

The  problem  of  modeling  multiple  time  series  is  one  on 
whose  theory  I  have  written  extensively  in  Parzen  (1967) ,  (1967a) 
(1969),  (1977),  and  Parzen  and  Newton  (1900).  I  would  like  to 
show  how  results  and  notation  from  these  papers  help  us  to 
derive  and  clarify  the  results  presented  by  Geweke. 

Let  X(t)  and  Y(t)  be  multiple  time  series,  with  zero  means, 
jointly  normal,  and  jointly  covariance  stationary.  To  study 
the  relations  between  X(‘)  and  Y(-).  one  models 


The  covariance  matrix  R(v)  ■  E[Z'(t)  Z(t+v)]  is  assumed 
to  be  summable  so  that  the  spectral  density  matrix 

f (w)  -  I  e-2nivw  R(v) ,  0<w<l , 

1/m— co 

exists.  Then  R(v)  -  ^e2ltlvw  f(w)  dw. 
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The  joint  covariance  and  spectral  density  matrices  of  X  and  Y 
are  described  by  the  blocks  in  the  partitioned  matrices 


R(v)  »  |  Rxx(v)  RXY^v^ 

RYX^v^  RYY^v^ 


f(w)  =  I  fxx(w)  fXY(w) 
fYX^  fYY^w^ 


Autoregressive  analysis  models  Z(t)  by  a  joint  infinite  order 
autoregressive  scheme: 


"x(tf 

X(t- 

-1) 

X(t-m) 

=  Ay  V  (1) 

*4".  •  .4*  A«  y (nO 

Y(t) 

X,  I 

Y(t- 

•1) 

A,  I 

Y(t-m) 

aXyU) 


n(t) 


+  n(t), 

Aj^Cj)  Axy(j) 

AYX(j)  AYY(j) 

nx  (t)~]  joint  innovations 
nY  (t) 


A  preferred  notation  for  E  is  E(X, Y|x“ ,Y") . 

We  call  n(t)  the  joint  innovations,  and  E  the  Joint 

innovation  covariance  matrix.  We  can  define  n(t)  «*  infinite 
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memory  prediction  errors 

n(t)  =  / X(t) \“E[/k(t)  ! 


X(t)  \"E  /X(t)| 
Yd)/  1\Y(D/ 


X ( l: - 1 )  ,  Y(t-l)  ,  . .  .X(t-in)  ,Y(t-in)  , 


The  joint  innovations  should  be  contrasted  with  the 
individual  innovations 

X  (t)  =  X(t)  -  E [X( t ) | X(t-l) . X(t-m) , . . . J 

Y  (t)  =  Y(t)  E[Y(t) | Y(t-l) , . . . , Y(t-m) , . . .] 
which  provide  individual  infinite  order  autoregressive  models 


X(t)  =  Ax|x(1)  X(t-1)+. . .+AX|X  X(t-m)+. . .+X(t) 

Y(t)  =  AY|Y(1)  Y(t-1)+. . .+Ayjy  Y(t-m)+. . .  +  Y(t) . 

The  individual  innovation  covariance  matrices  are  denoted 

=  £(X|X")  -  E[{X(t)}'  X(t)J , 

ZY  -  S(Y | Y“>  -  E [ {Y(t) } '  Y(t) ] . 

The  innovation  innovations  are  defined  to  be  the  joint 
innovations  of  the  joint  time  series  rx(t)*j  of  individual 

L*<t>J 

innovations.  A  remarkable  theorem  is  that  the  innovation 
innovations  are  identical  with  the  joint  innovations.  Thus  in 
practice,  one  can  determine  the  joint  innovations  of  X(t)  and 
Y(t)  by  first  "prewhitening"  them  to  form  X(t)  and  Y(t),  whose 
joint  innovations  are  then  determined. 


The  general  theory  of  multiple  time  series  discussed  below 
is  phrased  in  terms  of  general  stationary  time  series  X(t)  and 
Y(l).  However  it  works  best  in  practice  when  applied  to  time 
series  which  have  been  somewhat  pre-whitened. 

In  several  papers  on  model  identification,  Parzen  shows 
that  the  individual  innovations  of  a  time  series  are  essentially 
unique,  while  the  whitening  filter  which  generates  them  may  be 
expressed  in  diverse  ways  as  a  series  of  filters  representing 
detrending,  deseasonalizing,  and  innovations  operations. 

To  model  Y(t)  we  compare  the  properties  of  the  prediction 
errors,  and  prediction  error  covariance  matrices,  corresponding 
to  five  sets  of  explanatory  variables: 

Prediction  error  Covariance  Matrix 

(YV | Y_) (t)  =  Y(t)  -  E [ Y ( t) | Y(t-l) . Y(t-m) ]  Z(Y|Y‘) 

(YV|X",Y")(t)  -  Y(t)  -  E [Y(t) | X(t-l) . X(t-m) ,  E(Y|X",Y") 

Y(t-l) , . . . , Y(t-m) ] 

(YV|X+,Y')(t)  =  Y(t)  -  EtY(t) |X(t) ,X(t-l) . X(t-m) ,  E(Y|X+,Y') 

Y(t-l) . Y(t-m)  ] 

(YV|X)(t)  -  Y(t)  -  E [Y(t)  | X(s)  ,  -oo<s«»]  E(Y |X) 

(Yv|X,Y')(t)  -  Y(t)  -  E[Y(t)|X(s),  -«<s«,  Z(Y|X,Y") 

Y(t-l)  ,  Y(t-2) . Y(t-m)  ] 

It  should  be  noted  that  after  a  conditioning  sign  | ,  X“ 
represents  the  past  of  X,  X+  the  past  and  present  of  X,  and  X 
the  past,  present,  and  future  of  X. 
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The  spectral  densities  of  the  various  error  series  are 
denoted  fyV|y-(w),  fYv  |  X"  ,  Y"  ^  ’  fYv|X+,Y"^’  fYv|X^' 

f  Yv  |  X,  Y~  (w)  • 


(1)  Y  |  Y~ 


E(Y|Y~)  is  the  individual  innovation  covariance 


matrix  Ey;  (Yv|Y”)(t)  is  white  noise. 

(2)  Y 1 X~ ,  Y~ :  E (Y I X~ ,  Y')  is  the  block  Eyy  in  the  joint 
innovation  covariance  matrix;  (YV|X~,  Y  ) (t)  is  white  noise. 

(3)  Y|X+,  Y~ :  Parzen  (1967)  shows  (p.  401) 


(YvlX+,Y‘)(t)  =  (Yv|X\Y")(t)  -  Lyx  Exx‘a  (Xv|X',Y')(t); 
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E(Y|X+,Y")  =  Eyy  -  Eyx  Ej^' 


XY 


(4)  Y | X:  Parzen  (1967)  shows  that  a  joint  autoregressive 
model  for  X(t)  and  Y(t)  provides  f(w)  from  which  one  can 
compute  the  statistical  parameters  of  the  representation 


by 


Y(t)  -  (Yy|X)(t)  +  (Yv | X) (t) , 

(Yg|X)(t)  -  B(L)  X  (t),  B(z)  -  ?  B(k)zk 

k“-“ 


B(e27,iw)  -  fyx(w)  ^^(w) 

fyv |X(W)  m  fyy(w)  *  fyx(w)  fxx  (w)  fxy(w) 

E(Y|X)  -  ^lfYv|X^  dw* 

In  practice,  to  estimate  the  time  domain  coefficients  B(k) 

a  O  iw  4 

from  an  estimator  B  (e  )  one  uses  regression  methods. 
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(5)  Y | X ,  Y~ :  As  Geweke  shows,  find  an  autoregressive  model 
for  (Y^|X)(t); 

gyv(x(L)  (Y v  | X)  ( t)  =  c(t). 

It  generates  a  white  noise  sequence  r(t)  which  can  be  identified 
with  (Yv |X,Y~) (t) .  Further  a  model  for  Y(t)  as  a  function  of 
X(s)  ,  and  Y(s),  s<t-l  is  given  by 

gYV|X(L)  V(t)  =  gYv(x(L)B(L)  X(t)  +  e(t) ; 

E(Y|X,  Y  )  =  *  covariance  matrix  of  e(t). 

The  time  domain  coefficients  of  the  filter  with  input  X(t) 
have  Fourier  transform 

„  ,  -2TriwN  n,  -2iTiWv 

gYv|X  '  B(e  ' ‘ 

If  one  needs  only  the  log  determinant  of  it  can  be  calculated 
without  fitting  an  autoregressive  scheme: 

log|£c  |  -  J1  log|fy\>|X(w)|  dw. 

One  need  not  actually  calculate  the  spectral  density  since 
Geweke 's  Theorem  1  shows  that 

In  |  £ (Y | X, Y_) |  -  fn|£(X,Y|X“,Y") |  -  In  |Z(X|X‘)| 

The  meaning  of  the  various  definitions  of  feedback,  and  the 
formulas  for  them  given  by  Geweke' s  Theorem  1,  is  easily 
understood  if  one  employs  the  notation  we  have  introduced. 


V 


Measure  of  linear  dependence  (or  information)  y  „ 

=  In  det  >:  (X  |  X")  -  In  det  J:(X|X~,Y) 

=  tn  det  X(Y|Y")  -  In  det  E(Y|Y",X) 

Measure  of  instantaneous  linear  feedback :  • Y  = 

=  in  det  >;(X|X",  Y")  -  in  det  i(X|X',Y+) 

=  tn  det  T.( Y|X',Y')  -In  det Z(Y|X+,Y") 

Measure  of  linear  feedback  from  Y  to  X:  * 

=  tn  det  E(X|X“)  -  tn  det  £(X|X“,Y") 

=  tn  det  E(Y|X+,Y")  -  tn  det  Z(Y|X,Y“) 

Measure  of  linear  feedback  from  X  to  Y:  F^y  * 

=  tn  det  E (Y | Y")  -  In  det  E(Y|Y',X") 

=  tn  det  K(X|X_,Y+)  -  tn  det  E(X|X‘,Y) 

Theorem  1  in  Geweke's  paper  shows  that  there  is  a  crucial 

identity  from  which  the  equivalence  of  the  foregoing  definitions 

follows  immediately: 

In  det  E(X|X-,Y+)  +  tn  det  Z(Y|X",Y") 

-  In  det  E(X|X~ ,Y)  +  tn  det  E(Y | Y-) 

=  In  det  E  (X,  Y  |  X-  ,  Y~) 

These  feedback  measures  seem  to  me  to  be  most  clearly 
interpreted  as  measuring  the  significance  of  various  variables 
as  independent  variables  in  a  model  for  Y(t). 
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Variables 

In 

Variables  tested 

Measure 

in  Model 

Words 

for  inclusion 

Add 

FX  >Y 

Y" 

Past  Y 

Y'.X~ 

Past  X 

t 

F  y  y 

Y“,X" 

Past  Y, 

Y"  X+ 

Present  X 

X  *  I 

Past  X, 

> 

fy>x 

Y',X+ 

Past  Y, 
Past  X, 

Y'.X 

Future  X 

Present 

X 

fx,y 

Y” 

Past  Y 

Y",X 

All  X 

This  table  also  exhibits  the  information  decomposition  of  the 
identity: 

FX,Y  FX+Y  +  FX-Y  +  FY-X 

One  uses  y  to  compare  the  hypotheses  Hq:  model  Y  by  Y  ; 

:  model  Y  by  Y~,X.  In  addition  one  should  compute  E(Y|X) 
in  order  to  compare  the  hypotheses  Hq:  model  Y  by  X;  H^:  model 
Y  by  X,Y‘. 

In  any  empirical  multiple  time  series  analysis,  one  should 
compute,  and  report,  F.£,  Ly,  l  (the  individual  and  joint  innovation 
covariances) .  Then  one  should  compute  (and  test  for  significant 
difference  from  zero) 

FX-*-Y  “  EYY 

F^.y  =  “  frt  det  (I  “  Eyy  £yx  ^^CX  Ij^y) 

=  In  det  Fjj  +  In  det  Z^  -  In  det  I 

Fy>x  ”  In  det  Z ^  -  In  det 

The  computation  of  these  determinants,  and  additional 
insight  into  the  relations  between  variables,  could  be  attained 
by  computing  the  eigenvalues  and  eigenvectors  of  the  matrices, 


such  as  y"1  (Y  |  Y' ,  X")  >:(Y  |Y"  ,X+)  ,  whose  log  determinants  are 
being  calculated.  The  eigenvalues  can  bo  Interpreted  in  terms 
of  various  canonical  correlations  (see  Parzen  and  Newton  (19/9)) 

Finally,  to  estimate  (from  data)parameters  such  as  T, , 

>^,  and  y y .  it  is  strongly  recommended  that  one  use  approximatin 
autoregressive  schemes,  and  order-determining  criteria  such  as 
CAT  (see  Parzen  (1974),  (1977)). 

As  with  any  excellent  piece  of  research,  Geweke’s  paper 
raises  many  open  questions,  some  of  which  have  been  alluded  to 
in  my  discussion. 

The  definition  of  the  feedback  spectral  measure  fy-t-X^w^ 
given  by  Geweke  is  impressive.  Only  experience  can  show  us 
whether  it  should  be  routinely  computed  in  empirical  research; 
it  may  suffice  to  use  as  the  feedback  spectral  measure 

f^  y X  —  f y y  ( W )  ^XX  ( ^ )  ^XY^^^ 

computed  using  canonical  spectral  analysis  (see  Brillinger 
(1981) ,  chapter  10  for  references  to  this  literature) . 


-  I  0- 


REFERENCES 


Brillinger,  David  R.,  (1981)  Tunc  Series:  Data  Analysis  and 

Theory,  Expanded  edition.  Holden  Day:  San  Francisco. 

Parzen,  E. ,  (1967)  "Empirical  mulLiple  time  series  analysis" 

Proc.  of  the  Fifth  Berkeley  Symposium  on  Mathematical 
Statistics  and  Probability,  edited  by  L.  LeCam,  and  J . 
Neyman,  University  of  California  Press,  1967,  Vol.  I, 
pp.  305-340. 

Parzen,  E.,  (1967a)  "Time  series  analysis  for  models  of  signal 
plus  white  noise:  in  Spectral  Analysis  of  Time  Series, 
edited  by  Bernard  Harris,  Wiley:  New  York,  1967 ,  233-257 . 

Parzen,  E.,  (1969)  "Multiple  time  series  modeling"  Multivariate 
Analysis  -  11,  edited  by  P.  Krishnaiah,  Academic  Press: 

New  Yorl<,  389-409. 

Parzen,  E.,  (1974)  "Some  Recent  Advances  in  Time  Series  Modeling" 
IEEE  Transactions  on  Automatic  Control,  AC- 19 ,  723-730. 

Parzen,  E .  ,.  (1976)  "An  Approach  to  Time  Series  Modeling  and 

Forecasting  Illustrated  by  Hourly  Electricity  Demands" 
Technical  Report  (Statistical  Science  Division,  SUNY  at 
Buffalo) . 

Parzen,  E. ,  (1977)  "Multiple  Time  Series:  Determining  the 

Order  of  Approximating  Autoregressive  Schemes"  Multivariate 
Analysis  -  IV,  Edited  by  P.  Krishnaiah,  North  Holland : 
Amsterdam,  253-295. 

Parzen,  E.,  (1979)  "Forecasting  and  Whitening  Filter  Estimation" 
TIMS  Studies  in  the  Management  Sciences,  12,  149-165. 

Parzen,  E.  and  Newton,  H.  J.,  (1979)  "Multiple  Time  Series 

Modeling,  II"  Multivariate  Analysis  -  V,  edited  by  P. 
Krishnaiah,  North  Holland:  Amsterdam,  181-197. 


